REDEEM:ITS Authoring Tools and Human Teaching

REDEEM depends on human teaching tactics and strategies. Its design is predicated on the view that teachers should be provided with the means to express their theories about how learners should be taught and that ITSs should then teach students according to these theories. In this paper, we describe the nature of this dependence, what functionality this brings and what in turn we can learn about human teaching through its application in REDEEM. REDEEM is an ITS authoring environment which allows teachers to take existing computer-based teaching material (CBT) and turn it into a simple ITS that delivers differentiated and adaptive instruction to their class. The underlying CBT is enhanced by providing alternative sequences through the course and variations in the course content, a range of questioning and interaction strategies, and a variety of teaching styles that concern such aspects of teaching as provision of help, student control, difficulty and position of questions. As a result, a single course can be adapted to suit the needs of a variety of different learners and functions. In order to achieve this, the REDEEM tools are based upon the traditional ITS decomposition of the teaching process into the domain model, the student model and the teaching strategy model. In essence, teachers describe what they teach, who they teach and how the teach it. For example, they describe the difficulty of course components and the relationships between them, they classify students in their class by placing students into author-defined categories and they create teaching strategies by manipulating dimensional sliders of teaching tactics which vary such factors as the degree of student control and position/amount of questions (See Figures 1 & 2). If the teacher chooses, REDEEM can take a more active role and monitor students performance. For example, students doing well could be reclassified and they would receive the teaching strategies and content appropriate for that category. More details of the environment can be found in [6] REDEEM has only a selection of the range of strategies that human teachers use. It performs no natural language processing, captures only a very limited range of actions from the student and has no deep knowledge of the domain. But, in other ways it does attempt to mimic human teaching. Human teachers employ a range of strategies that they suit to individual students adapting them if their perceptions of a student change [4]. This has been identified as fundamental if teachers are to use ITSs in the classroom [5]. REDEEM also incorporates strategies derived from human teaching such as contingent help [7]. So, while the elements of the teaching strategies are those suitable to a simple machine teacher, the general approach is human inspired and human dependent. Fig 1. Authoring domain content Fig 2. Authoring teaching strategies By allowing teachers to incorporate their own pedagogic beliefs, we hope to see a number of advantages. First, the primary motivation informing REDEEM is to provide teachers with the tools to customize ITSs to the perceived needs of their students, in the expectation that these students will learn more efficiently or effectively. Second, REDEEM allows the same course material to be reused in a variety of situations. This is achieved either by adapting the ITS for different types of learners or different types of function (whole class teaching, individual learning, practise and revision etc). Third, REDEEM can be used as an experimental laboratory. Many different teaching strategies can be applied to the same course material allowing predictions to be tested concerning the effectiveness of these alternative approaches. These strategies can either be informed by theory or can be generated by teachers. Finally, we hope that by using REDEEM to express their pedagogic theories teachers can reflect upon the nature of these theories. This can occur either through the process of externalizing implicit assumptions, by acting as a student in the ITS shell and experiencing the consequences of these decisions, or by comparing different teachers' views of the same material. Our current studies are aimed at evaluating the effectiveness of this approach to ITS development. Of particular relevance to this workshop are two recent studies aimed at determining if teachers find that REDEEM can incorporate their own pedagogic beliefs. In a sense we are addressing a related plausibility problem to the one discussed by Lepper [3] and the AI-Ed 99 panel that is, whether teachers will accept an ITS more readily if it teaches their students in a way that corresponds to their own view of teaching, and whether they can recognise their view of teaching when interpreted by a computer. If the results of these studies prove successful the next stage in the research will be to explore the learning outcomes of students using an ITS created for them by their class teacher. Studies reported in [1,2] examined whether REDEEM can effectively capture and then express teachers' theories about who, how and what to teach. Four educators were asked to author the same course (Understanding Shapes). Three were classroom teachers and one was a teacher trainer in primary mathematics and so all were familiar with the content of the course. In order to compare their authoring they were provided with a class of virtual children whose descriptions were provided by the experimenters. These had previously been standardized with the help of a local headteacher. We asked these authors to describe the content of the course material (e.g. to create sections in a flat unstructured course, to author questions, suggest reflection points and non-computer tasks, and to describe each page in the course along a number of relevant dimensions such as familiarity, difficulty of material etc). Then authors categorized the students in the virtual class along any dimensions they wished, and created different teaching strategies by combining different values of the strategy dimensions and identifying relevant questions. Having done this, authors associated sections of the course with student categories and assigned teaching strategies to student categories. To examine if teachers would accept courses authored by others we presented two of the educators with previously authored domain content but no authoring on student categorization or teaching strategies. The other two authors authored the ITS from scratch In total, including time to familiarise themselves with REDEEM and the course material, authoring took between 7 and 12 hours for a course that takes around six hours to deliver. We found that the authors shared very similar views of who they were teaching. Each author created a number of student categories and assigned different students to each one. This provided support for the view that teachers desire computer-based instruction that is adapted to the needs of different types of students. When comparing similarities between the different authors, we found that each created five categories of children and tended to place the same children in the same category. However, two authors used categories based upon both familiarity with the material and aptitude scores and two considered only aptitude. Further work will determine if these findings generalize to larger (virtual) classes, to classes known to the participants, and to teachers of other age groups. There were strong inter-author differences in what was taught as well as in the order in which it was taught. For example, the authors did not identify the same number of sections in the course. They constructed between 11 and 16 sections. There was basic agreement that the familiar 2d shapes should be taught first before more complex areas such as symmetry, tessellation and polygons are covered. But this generalisation hides variation. For example two teachers specified that circles should be taught before other basic shapes, another that it should come last, and one did not mind where it was taught. Certain concepts also seemed to provoke more disagreement. For example, the “tangrams” section was selected as the fourth, fifth, eighth or tenth section to be covered. Even when the created sections were alike, the ways that pages were structured within those sections could differ. This was more noticeable on the more complex topics. We are currently developing sequence analysis techniques in order to quantify such differences in course structure. Another feature of REDEEM’s course structuring tools is that teachers can impose their views of what different children should be taught by relating sections to student categories. This tool was very much appreciated by the authors and they tended to have very strong views on what material was appropriate for each student category. However, there was only limited consensus about these decisions. For example, “Alison” (categorised by all authors to be in the least advanced category) was assigned either 5/11, 10/13, 12/15 or 9/16 sections. As a result, even when authors hold similar views about the “correct” order to teach concepts, they may still differ in how many of these concepts are seen by each student. The authors also differed in how they taught these students. Each author created between two and six teaching strategies and assigned up to five of these to the student categories. These strategies were created by combining different aspects of eight different teaching dimensions. In total, the four authors used 20/24 of the possible options (although given the combinatorial nature of the tool, a tiny fraction of total combinations were used). This suggests that the teaching dimensions they were provided with were those that the authors saw as important for either differentiating their class (intra-author differences) or ones that they held different views from that the other authors (inter-author differences). In fact, authors tended to agree on how some dimensions of the teaching strategy should be used (e.g. teach general concepts before specific, when feedback upon performance should be given), but for other dimensions, such as such as degree of student control and whether answers to questions should be deduced or given, there were marked disparities between the authors. Taking all these design decisions together, it can be seen that the experience of using a REDEEM ITS for Understanding Shapes would have been very different for a child depending upon who was their teacher. They could have received different amounts of course content, structured in different ways, with variations in the position and amount of question, degree of control over their own learning, help provision etc. Furthermore, the designs of the ITSs were recognised by the authors as closely related to their instructional approach. All authors expressed more satisfaction with the course upon completion of authoring (although they also suggested some further changes) and this was also true of authors who were exposed to someone else's view of the course. Authors felt they would be more likely to accept their REDEEM ITS in their classroom than the CBT alone or another author’s ITS. We would argue that these studies address some key questions about the role of human teaching strategies in ITS. Firstly, they show that teachers are able to use REDEEM to create ITSs that more closely embody their pedagogic theories than a pre existing system. Secondly, differences between the authors show that there is no single agreed way of teaching students in a particular area authors expressed clear differences in the domain content and teaching strategies. There were noticeable similarities between the authors, but at the same time, the differences between them were striking enough to warrant the provision of ITS authoring tools. Hence, one lesson from this study is that there is no single “correct” human teaching tactic that an ITS should aim to emulate. This suggests a number of further developments. Teachers could create ITSs that reflect their own pedagogic preferences which may lead to greater acceptability and, hopefully, educational effectiveness in the classroom. It is also possible to test these educational theories. Currently, we have no views on whether any of the courses authored in this study would have led to significantly better learning outcomes. However, such decisions could be compared by running classroom trials in order to advise teachers and educational software designers about effective teaching strategies and course structures. It also suggests ways of helping teachers' professional development. They are given opportunities to externalise their pedagogic preferences and reflect upon them and they can compare other teachers’ views on how to teach the course. We conclude that even simple ITSs can benefit from human views of teaching.